# Efficient Quantization

DeepSeek-R1-Distill-Qwen-14B is an optimized large language model with a parameter scale of 14B, released by DeepSeek AI. It is distilled from the Qwen architecture and offers multiple GGUF quantization versions to improve performance.

A quantized version of Medra27B, offering multiple quantization types, suitable for multiple fields such as text generation and medical artificial intelligence.

A quantized version of the NVIDIA Llama-3.1-Nemotron-Nano-4B-v1.1 model, processed with llama.cpp tools for various quantization methods, suitable for running in resource-constrained environments.

This model has undergone self-quantization processing, with output and embedding tensors quantized to f16 format, and the remaining tensors quantized to q5_k or q6_k format, resulting in a smaller size while maintaining performance comparable to pure f16.

Llamacpp quantized version based on andrewzh's Absolute_Zero_Reasoner-Coder-7b model, supporting multiple quantization levels, suitable for reasoning and code generation tasks.

Qwen3-14B-AWQ is the latest 4-bit AWQ quantized version of the Qwen series large language model, supporting seamless switching between reasoning and non-reasoning modes with powerful inference, instruction following, and agent capabilities.

Quantized version of Qwen3-4B-abliterated, quantized using llama.cpp, supports multiple quantization types, suitable for text generation tasks.

A quantized version based on Qwen/Qwen3-1.7B, using llama.cpp for quantization, supporting multiple quantization types, suitable for text generation tasks.

A quantized model based on dreamgen/lucid-v1-nemo, processed with llama.cpp for various quantization levels, suitable for text generation tasks.

This model is a GGUF format version converted from mlabonne/gemma-3-4b-it-abliterated, suitable for local operation and inference.

Gemma 3 12B is a large language model that provides a quantized version in GGUF format, suitable for local deployment and use.

The EXAONE Deep series models excel in reasoning tasks such as mathematics and programming. This model is the 4-bit AWQ quantized version with 2.4 billion parameters

Gemmasutra-Small-4B-v1 is a 4B-parameter text generation model, quantized based on llama.cpp, suitable for various quantization version choices.

InternVL2_5-4B-AWQ is the AWQ quantized version of InternVL2_5-4B using autoawq, supporting multilingual and multimodal tasks.

Quantized version based on ozone-ai/0x-lite model, supporting Chinese and English text generation tasks, using llama.cpp for imatrix quantization, offering multiple quantization options to adapt to different hardware requirements.

This is a quantized version based on TheDrummer/Gemmasutra-9B-v1.1 model, processed using llama.cpp, suitable for text generation tasks.

This model is a multilingual text generation model converted from bigscience/mt0-xxl-mt to GGUF format via llama.cpp, supporting various language tasks.

An 8B-parameter summary generation model optimized based on Llama3 architecture, offering multiple quantization versions

FLUX.1-schnell is an efficient text-to-image generation model based on a diffusion model architecture, supporting English text input to generate high-quality images.

Phi-3.5-mini-instruct_Uncensored is a quantized language model suitable for use under various hardware conditions.

Quantized version of FLUX.1-schnell, a text-to-image diffusion model supporting multiple quantization precision options

This model is a sentence similarity model converted from BAAI/bge-m3 to GGUF format using llama.cpp via ggml.ai's GGUF-my-repo space.

Chronos is a family of pretrained time series forecasting models based on language model architectures, trained by quantizing and scaling time series into token sequences.

Chronos is a family of pre-trained time series forecasting models based on language model architecture, which transforms time series into token sequences for training to achieve probabilistic forecasting.

LLaVA 1.6 34B is an open-source multimodal chatbot model developed by fine-tuning a large language model on multimodal instruction-following data. It supports image-to-text and text-to-text generation tasks.